Positive False Discovery Proportions: Intrinsic Bounds and Adaptive Control

A useful paradigm for multiple testing is to control error rates derived from the false discovery proportion (FDP). The false discovery rate (FDR) is the expectation of the FDP, which is defined to be zero if no rejection is made. However, since follow-up studies are based on hypotheses that are actually rejected, it is important to control the positive FDR (pFDR) or the positive false discovery excessive probability (pFDEP), i.e., the conditional expectation of the FDP or the conditional probability of the FDP exceeding a specified level, given that at least one rejection is made. We show that, unlike FDR, these two positive error rates may not be controllable at a desired level. Given a multiple testing problem, there can exist positive intrinsic lower bounds, such that no procedures can attain a pFDR or pFDEP level below the corresponding bound. To reduce misinterpretations of testing results, we propose several procedures that are adaptive, i.e., they achieve pFDR or pFDEP control when the target control level is attainable, and make no rejections otherwise. The adaptive control is established under a sparsity condition where the fraction of false nulls is increasingly close to zero as well as under the condition where the fraction of false nulls is a positive constant. We demonstrate that the power of the proposed procedures is comparable to the Benjamini-Hochberg FDR controlling procedure.